Virtualization provides datacenters with highly efficient and available resource, networking, and storage management to reduce infrastructure costs such as capital, power, space, cooling, labor, and the like. As workloads continue to move to virtualized environments, some existing platforms fail to scale to handle the additional workloads. The existing platforms may exhibit suboptimal performance when user-level operations are scheduled poorly. For example, naïve scheduling of the user-level operations can overload the virtual datacenter and slow down the productivity and overall performance of the platform. That is, the performance (e.g., throughput) of these platforms may degrade when the system management overhead grows to a point where the platform as a whole runs out of resources. The resources may be exhausted due to physical resource constraints (e.g. processor, memory, and communications) or logical resource constraints (e.g., locks, semaphores) in any of the components of the platform. In such scenarios, not only does throughput degrade, but also the platform becomes unpredictable and unstable because the platform is running beyond its boundaries.
The existing platforms lack a mechanism to inform clients that the platform is overloaded. As such, some of the clients may implement static throttling schemes to impose artificial task limits. However, these static throttling schemes tend to be overly conservative, thus reducing performance of the clients and keeping the platforms from operating optimally.
Further, the platforms are constantly in flux due to the wide variety of operations being executed by varying amounts of clients and users. As such, the platforms do not have a fixed configuration for optimal performance, making it difficult to operate optimally. Rather, the set point for optimal performance in these platforms varies dynamically based on the type of workload, number of users issuing tasks, and other factors.